Multivariate Time Series Prediction via Temporal Classification

One of the important problems in many process industries is how to predict the occurrence of abnormal situations ahead of time in a multivariate time series environment. For example, in an oil refinery, hundreds of sensors (process variables) are installed at different sections of a process unit. These sensors constantly monitor the development of every stage of the process. Typically, each process variable (represented by a sensor) is given some value ranges, e.g., low, normal, and high. When the value of a variable reaches an undesirable range, e.g., low or high, an alarm is generated and displayed on a computer screen. The human operator, who monitors alarms, uses his/her experiences to decide what alarms are important and what actions to take. This manual process is acceptable for those non-critical variables. When alarms from these variables are generated, it is not too late to react. However, there are some critical variables that can significantly affect the quality of the product or indicate danger in the process. When alarms are generated from these variables, it is often too late to perform corrective actions. This can result in poor product quality, or even shutdown of the whole plant to prevent fire or explosion. It is thus of great importance to predict when these critical variables will reach undesirable ranges ahead of time. By doing so, the plant engineers can have sufficient time to take preventive actions. Traditionally, engineers and researchers tried to build mathematical models to understand the physical process and to capture the temporal relationships among the variables. However, in a typical plant, there are hundreds of variables, and their interactions are extremely complicated. Building an accurate mathematical model is very difficult, if not impossible. So far, little success has been achieved. Although time series forecasting techniques exist in statistics, they are mostly for a single time series. Those that work with multiple time series often do not produce satisfactory results. In the past few years, there were a number of reported studies in data mining on pattern discovery, and pattern matching in time series data. However, discovered patterns are not necessarily useful for prediction. Limited work has been done on how to build predictive models directly from multivariate time series. In this paper, we study a special form of time series prediction, i.e., the prediction or dependent variable taking discrete values. Although in a real application, this variable may take numeric values, the users are usually only interested in its value ranges, e.g., normal or abnormal, not its actual values. This problem is related to traditional classification. In traditional classification, a classifier is built from the training data, which is then used to predict the class of a new data object. Many successful techniques were produced in the past. However, these techniques cannot be directly applied to time series data because they are unable to consider time in their model building processes. In this work, we extended two traditional classification techniques, namely, naïve Bayesian classifier [1] and decision trees [3] to suit temporal prediction. This results in two new techniques, temporal naïve Bayesian model (T-NB), and temporal decision tree (T-DT). T-NB and T-DT have been tested on 7 real-life datasets from an oil refinery. Experiment results show that they perform very accurate predictions. Our contributions: To the best of our knowledge, this is the first time that the two classic classification techniques have been extended for time series prediction. Unlike traditional classification, the key issue in temporal predication is that one variable’s fluctuation often causes changes in other variables after certain periods of time, i.e., with some time lags or delays. Thus, we augment the existing classification methods with a temporal component, which is able to find a suitable prediction time lag (or delay) for each variable at different stages of the model building. We have tested the proposed T-NB and T-DT methods and compared them with existing techniques using 7 real-life datasets. The results show that they perform far better predictions than existing statistical and other types of techniques. The results also show that the T-NB method produces more accurate predictors than the T-DT method. The T-NB method is also very efficient. It only scans the data twice, and allows the data to reside on disk. Further details can be found in [2].